Deterministic Dynamic Programming

The terminal set M is a closed subset of Rn+1. The admissible control set U is assumed to be the set of piecewise continuous function on [t0, t1]. The performance function J is assumed to be C1. The function V (·, ·) is called the value function and we shall use the convention V (t0, x0) = ∞ if the control problem above admits no feasible solution. We will denote by U(x0, t0), the set of feasible controls with initial condition (x0, t0), that is, the set of control u such that the corresponding trajectory x satisfies x(t1) ∈ M . Proposition 1 Let u(t) ∈ U(x0, t0) be a feasible control and x(t) the corresponding trajectory. Then, for any t0 ≤ τ1 ≤ τ2 ≤ t1, V (τ1, x(τ1)) ≤ V (τ2, x(τ2)). That is, the value function is a nondecreasing function along any feasible trajectory.